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In support of new human rated spacecraft development programs, the Mission Operations 
Directorate at NASA Johnson Space Center has implemented a formal method for the 
assessment of spacecraft operability. This “Spacecraft Flight Operability Assessment Scale” 
defines six key themes of flight operability, with guiding principles and goals stated for each 
factor. A standardized rating technique provides feedback that is useful to the operations, 
design and program management communities. Applicability of this concept across the 
program structure and life cycle is addressed. Examples of operationally desirable and 
undesirable spacecraft design characteristics are provided, as is a sample of the assessment 
scale product. 


I. Introduction 

T HE design of a human rated spacecraft is a complex and costly process requiring the integrated assessment of 
many individual criteria. Historically, it has been difficult to include in that integrated assessment the design’s 
full impact on the flight operations community. The unique “operability” requirements have not been well 
understood, nor has there been a well-defined set of criteria for assessing operability. As programs approach their 
operational phases, program managers and flight operations organizations alike are often surprised when faced with 
difficult and costly operations implementations. A formal means of forecasting operability issues during the 
development phases of a program is therefore necessary to reduce operations phase costs. 

The challenge in addressing flight operability needs for a new program is threefold: (1) there is no accepted, 
universal definition of flight operability, (2) there is no clear mapping of flight operability needs to program and 
vehicle requirements, and (3) there is no formal method to assess flight operability characteristics given a spacecraft 
design and mission definition. Development of a practical flight operability assessment methodology requires the 
establishment of several key items. Flight operability itself must be defined in terms that are relevant both to the 
flight operations community and to program management. Specific operability goals must be set, preferably as 
formal design and performance requirements. Objective measures must be established to determine compliance 
with those requirements. 

Several organizations have attempted to define flight operability and specific associated design requirements. 
The European Space Agency (ESA), NASA Jet Propulsion Laboratory (JPL), and NASA Lyndon B. Johnson Space 
Center (JSC) have all published documents intended to better specify operability needs. ESA’s Space Segment 
Operability Standard addresses robotic spacecraft operations safety, efficiency, and cost effectiveness but 
recognized the difficulty in defining clear criteria for onboard automation capabilities. 2 NASA JPL defined a 
similar set of design criteria for its robotic spacecraft. To document similar lessons learned for human spaceflight, 
NASA JSC published its own Space Systems Operational Design Criteria Manual. 3 Each document provides a 
valuable resource of design suggestions, but none fully encompasses the needs for human rated spacecraft nor 
provides a clear process for the evaluation of real system designs against documented recommendations. 

The challenges associated with establishment of this methodology are not unlike those faced by the aircraft flight 
test community in the 1950’s and 1960’s. Over the course of twelve years, a technique for the characterization of an 
aircraft’s handling techniques - as assessed by the pilot operating the aircraft - was developed by George Cooper 
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and Robert Harper. 4 The Cooper-Harper Scale has long stood as the standard tool for aircraft handling assessment. 
Modified versions of this scale are employed widely in the assessment of crew equipment and interfaces for NASA 
human rated spacecraft. 

In response to these challenges, the Mission Operations Directorate at NASA JSC has established a formal 
method for the evaluation and communication of a spacecraft system design’s operational characteristics. The 
spacecraft flight operability assessment method described in this paper is born of the desire to identify the operations 
systems drivers and critical requirements that are a significant influence on operations cost, schedule, performance 
and risk. This process is not intended to replace or replicate other critical assessments such as risk, reliability or 
safety assessments. Instead, this new technique adds to a program’s assessment toolset a means to address the 
concerns and potential cost drivers that are unique to the operational phase of a program and the flight operations 
community. 


II. Flight Operability Definition 

Apollo 11 flight director and former director of NASA JSC’s Mission Operations Directorate Eugene Kranz 
defined the flight operations infrastructure as a system designed to “Maximize mission success, to minimize risks to 
the [vehicle] and the crew, to decrease operating costs, and to achieve an effective balance in the application of all 
operational resources.” 5 In this context, the measure of a system’s flight operability is the measure of the degree to 
which that system enables a balance of maximum mission success, minimal risk, and minimum operating cost. 
Because the flight operations community is held to the highest standards of safety and mission success, the most 
variable of these factors is typically operating cost. 

Any measure of flight operability must encompass the impact on cost, responsiveness and risk incurred in safely 
executing intended operations with a spacecraft as designed and manufactured. Cost is driven by both the 
developmental investments required to build the operations infrastructure (facilities, operations techniques and 
products, and trained personnel prepared to execute operations), by the recurring cost of maintaining that 
infrastructure and by the expense of executing mission planning, training and operations over the entire operations 
phase. Responsiveness reflects the duration over which an operation must be planned, reviewed and executed. 
Excessive time requirements reduce the availability and responsiveness of operations. Risk is the likelihood of 
success or failure of the operation. Additional consideration of risk must be given in the case that a failure endangers 
crew health, vehicle integrity or mission success. “Operations Integration” is the practice of weighing and balancing 
these factors. 

Flight operability is not only a function of the vehicle design, but also the mission requirements that the system 
must support. Therefore, a given system design may have different operability “scores” for different types of 
mission scenarios and operations. Consider a vehicle designed solely to achieve and maintain Low Earth Orbit may 
exhibit significant propellant margin in performing that mission. That same vehicle design may provide little or no 
margin if the mission is changed to achieve and maintain a lunar orbit. Therefore, a complete measurement of flight 
operability begins with the definition of the system or vehicle under study, the specific mission class, or mission 
phase, or more detailed operational scenario in which operability is to be assessed. For that specific set of design 
and mission conditions, an operability assessment must identify and objectively assess the key influences that 
impact flight operations ability to meet safety, mission success and operating cost constraints. 

III. Programmatic Impacts of Flight Operability 

Although flight operability issues may be most apparent to those who execute mission planning, training, and 
real-time support activities, the impact of these issues can span across an entire program infrastructure. Spacecraft 
with low operability characteristics force the program, vendor, and operations communities to pursue complex 
tradeoffs between cost categories. Conversely, spacecraft with high operability characteristics do not require 
significant program-level operations cost tradeoffs. Consider an operability issue associated with a hazardous 
condition or operation. To reduce risk of a life threatening hazard, additional time and resources are spent in the 
analysis, planning, practice and execution of special procedures that mitigate the hazard. These are collectively 
referred to as “operational workarounds.” Failure to perform such analysis, planning, practice and diligence in 


4 The Use of Pilot Rating in the Evaluation of Aircraft Handling Qualities, George E. Cooper and Robert P. Harper, Jr., April 
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5 Eugene F. Kranz, STS Flight Operations - Concept versus Reality. AIAA Shuttle Environment and Operations Conference II, 
November 1985. 
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execution results in greater risks should that the hazard be experienced. The implications and costs associated with 
this hazard and associated operational workarounds are categorized below. 

A. Engineering and real-time operations support cost impact 

Program sustaining engineering costs are incurred whenever operational workarounds are invoked. For example, 
the International Space Station was initially certified to fly in only a very narrow range of attitudes. However, 
operational needs mandated that the spacecraft be maneuvered to and through a much wider range of attitudes in 
response to testing needs, off-nominal conditions, and the constraints of other docked spacecraft. Thermal analysis 
of each alternate attitude for ISS operations became a continuous task for the program throughout the program’s 
operations phase. The result was the need for additional personnel and tools to perform the analyses needed to 
support each mission and operation. 

B. Program responsiveness impact 

Program and mission responsiveness are also reduced when operability is not provided. The constant need for 
new analysis, new operations product development, and verification of these new products can cause weeks- or even 
months-long delays, in addition to increased operating cost, when making even changes to mission execution. 

The late discovery of multiple concerns regarding the ISS solar arrays resulted in the addition of many 
conflicting operational constraints on the orientation of the arrays. The daily task of planning solar array positions 
quickly changed from that of a basic capability of the flight control team to an effort requiring the constant 
involvement of a much larger team including vendor and program office engineering analysts, the development of 
new tools and processes, and a still evolving set of data products. The additional processes required to handle these 
challenges necessarily impact the timeliness within which changes can be made. 

C. Mission success impact 

Program reliability is reduced when adequate operability is not provided. Dependence on operators to “close the 
loop” for basic spacecraft functional capabilities leaves open the potential for human error and, in the case of 
requirements levied on ground-based operators, ties spacecraft reliability to ground network and communication 
satellite asset reliability. Any break in the chain of facilities and services that enable the operator to implement an 
operations workaround negatively impacts the overall reliability of the integrated vehicle-ground support system. 

During early International Space Station (ISS) operations. Mission Control Center operators were required to 
send hundreds of commands per day just to maintain ISS communication capability. These commands provide 
Tracking Data Relay Satellite System (TDRSS) pointing and selection data to onboard processors that are not able to 
adequately perform automatic selection. This left the vehicle prone to potential communications outages as a result 
of human error or onboard failures - that could impact science return. Again, an operational workaround could be 
provided, but the cost and risk associated with the workarounds had an impact on flight operations and the program 
at large. 


IV. Addressing Flight Operability Issues at the Program Level 

The Cooper-Harper Scale grew from a basic assessment tool to means of asserting formal operations-related 
performance requirements for aircraft designers. Over the span of a dozen years, the scale was refined and applied 
over a wider set of cases. Today, aircraft and spacecraft requirements documents typically specify minimum 
acceptable Cooper-Harper scores. This was the result of the general acknowledgment that the assessment technique 
added value, generated repeatable results, and that those results could be clearly and predictably mapped to design 
characteristics. 

Similarly, the definition of a formal technique for spacecraft flight operability should be viewed as an 
evolutionary process, beginning with a common definition of operability definitions and assessment criteria, but 
eventually reaching a common understanding that positively influences design requirements. A mature, program- 
endorsed operability assessment technique therefore can be integrated into several phases of a given program life 
cycle. The assessment methodology described in this paper is intended to provide a first step in the achievement of 
these goals. 

Operability expectations can be explicitly addressed in the development of program operations concepts, 
addressing the needs associated with mission planning, training, and execution activities. The requirements 
development process can be better informed through these operations concept details, and specific performance 
operations-driven requirements may be derived from the definition of operability criteria. Design activities and 
design reviews can benefit from operability guidelines as well as operability assessments that identify key issues 
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early in the design process. Finally, the same methods may be used to assess operability of changes and upgrades to 
be made during the operational phase of a program. 
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Figure 1. Flight Operability considerations throughout the program life cycle 


Formal flight operability assessment practices may be applied to both development and operational programs. For 
development programs, the Spacecraft Flight Operability Assessment Scale can and should be employed in 
generating inputs at formal design reviews (Subsystem Design Review, Preliminary Design Review, Critical Design 
Review) in less formal design team forums, and in the assessment of formal Change Requests (CRs). For 
operational programs, the scale may be applied to proposed incremental vehicle changes such as hardware upgrades 
and flight software updates. This includes the assessment of CRs, Problem Reports, and other notices that request or 
direct operational workarounds. 

Operability assessment techniques should not replace other critical evaluation methods such as operations 
concern or “watch lists,” Review Item Discrepancy (RID) submittals, hazard assessments or risk assessments. 
Similarly, operability assessments should not replace or replicate other critical assessment techniques that address 
safety, performance, and life cycle cost. Instead, operability assessment adds to those other methods by defining and 
assessing the factors that are unique to the flight operations community. 


V. Primary Themes in Flight Operability 

Review of the many individual recommendations of the flight operations community indicates six major 
operability themes - simplicity, margin, robustness, flexibility, situation awareness and control. These themes are 
discussed below. Note that, if not properly balanced, these operability themes can pose conflict. Features that make 
a system more robust may also make the system more complex. The judgment of subject matter experts must be 
applied to strike balance in these cases. 

A. Simplicity 

Simplicity - often referred to with its inverse, complexity - is the collective measure not only of the functions, 
interfaces and dependencies inherent in the system architecture, but also of the observations, decision and actions 
required of the human operator. The number and ease of operation of functions and interfaces in the operational 
environment drive the number and cost of analyses, tools, procedures, plans, constraints and training required. 
Simple systems that have few dependencies and few possible system configurations generally require fewer 
procedures, less training, and less effort to monitor and control. 

To address operability concerns, hardware and software should be as simple as practical, minimizing the number 
of unique interfaces, algorithms, and functions that require separate operational techniques to monitor and control. 
Functions and interfaces should be common and consistent, requiring a reasonable number of tasks and 
methodologies on the part of the operator. Tasks themselves should be simple, allowing the operator to concentrate 
on decisions to be made rather than detailed operational sequences to be performed. 

There are reasonable limits on the operationally desirable level of simplicity. A system that is so simple that it 
does not provide the flexibility or robustness to perform in off-nominal scenarios is not operationally viable. 
Careful consideration of the other operability factors should be included in an assessment of the appropriate level of 
simplicity in a system. 
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B. Margin 

Operational margin describes the amount of capability or consumable supplies available beyond that required to 
execute the mission. Operational margin provides assurance that the nominal mission may be safely executed and 
allows for continued operation in the event of unexpected conditions such as malfunction or mission scenario 
changes. 

There are three categories of operational margin: 

• Performance Margin - The ability of the system to provide greater capability than required for normal 
operation or in the event of any single failure. Measures of performance margin vary by vehicle subsystem. 
For example, performance margin for an electrical power system might be measured by power output 
capability while the measure for a communication system might be associated with the data bandwidth 
sizing. 

• Resource Margin - The amount of consumable commodities (propellant, atmospheric gases, stored energy) 
available beyond that required to support nominal flight operations. 

• Environmental Tolerance Margin - The system's ability to operate beyond the nominal operations 
environment for a given mission profile. 

Often, operational constraints and controls are required to ensure that adequate capability is available throughout 
a nominal mission and after an anomaly. These constraints and controls typically impact the ability to successfully 
complete all mission goals, as they limit the use of capabilities and resources even before an anomaly occurs. They 
also require the addition of more techniques, tools, products and training to the operations infrastructure. All of these 
additions result in increased life cycle cost. Margin is considered available for operational consideration only when 
formal analysis documentation of that margin is made available to the operations community. 

Lack of margin can have profound impacts on mission planning as well as real-time operations. More detailed 
pre-flight analysis must be performed to ensure that mission objectives may be met within the available resources, 
that the vehicle can perform required operations within its normal performance envelope, withstand potential 
anomalies, and that the flight environment does not exceed the vehicle’s limits. Lack of margin not only impacts the 
mission operations organization, but it also drives significant program sustaining engineering costs to provide 
additional case-specific analyses that support the flight operations community as well as program strategic planning. 

Flight systems should therefore provide margin in order to minimize operations constraints. Vehicle thermal, 
power, and communications capabilities should not be designed with operations constraints that result in the 
necessity for highly optimized mission timelines to accomplish normal operations such as rendezvous, proximity 
operations, and docking. Margin in all three of these categories is a significant driver in determining the amount and 
extent of mission- and activity-specific planning and analysis. Significant positive margins in key categories should 
be available in all mission phases. 

At the same time, excessive margin is not operationally desirable. For example, a system that provides resource 
quantities beyond any credible need may use so large a fraction of the allowable mass that fewer redundant strings 
are provided in the design. Expectations on available margin should be bounded by the maximum needs for an 
operational scenario (including off-nominal scenarios). In addition, care should be taken in scenarios that involve 
failure “stacking” (inclusion of multiple separate failure cases in one scenario). Credible failure scenarios include 
those that would allow continued mission execution and those that would initiate the abort or early termination of a 
mission. Failures after those that drive a mission abort or early termination are generally out of scope. 

C. Flexibility 

Flexibility is the ability of the system to accommodate change. This change can be to the mission scenario or to 
the vehicle configuration. When a system is inflexible, even small changes to the mission or vehicle configuration 
may require operational workarounds - additional tasks and responsibilities placed on operations personnel and 
facilities. Flexibility is generally defined by the system’s architecture. 

Flexible flight systems should be easily reconfigured or updated to account for new conditions and new 
capabilities during flight or between flights. Although this applies to both flight hardware and flight software, the 
impacts of inflexible software are the more acute. Operational experience often identifies necessary changes to 
limits, gains, and other parameters used by flight software. If recompilation of flight software is required to update 
such parameters, then these value updates will be costly and will require months or years to incorporate. 
Operational workarounds will be required for extended periods in order to account for discrepancies between the 
desired and provided values. 

There are reasonable limits to the desired degree of flexibility for an operable system. While some amount of 
flexibility is desired to allow for slight variation in mission profile and vehicle configuration, excessive flexibility 
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can result in additional operations challenges. Highly flexible systems may require more training, product 
development, and manual tending than is operationally desirable or affordable. 


D. Robustness 

Robustness describes the system’s ability to cope with changing conditions resulting from both nominal and off- 
nominal operations. Flight operations planning and analysis costs are often driven by the need to “protect” the 
system or vehicle from certain conditions and events. The nature and degree of these “protection” measures is 

determined by the system’s or vehicle’s robustness. Note that provisions such as performance margin and 

consumables margin is assessed in a separate “margin” category. The “robustness” category addresses redundancy, 
fault tolerance, cross-strapping and similar system architecture traits. 

To achieve operational robustness, flight systems should be designed to maintain fail operational capability (no 
loss of functionality after first failure); the design should ensure no single failure puts the mission in to a 
contingency. Systems should remain partially capable in off-nominal scenarios, allowing the continued use of 
remaining functionality without requiring significant operator action to recover that functionality. In many cases, 
cross-strapping - interconnections between components of two or more separate strings - are effective means for 
improving robustness in off-nominal scenarios. Redundant strings should be supported by separate data and power 
utility feeds to allow continued system availability after a single failure. 

No time-critical operator action should be required to prevent loss of mission, crew or vehicle. Time-critical 
operator actions are those that must be performed by a person within a limited time frame immediately following an 
event to ensure continued safe and effective mission execution. In general, the vehicle should automatically identify 
and reconfigure in response to failures that can impact mission success or crew/vehicle survival. Automated 
responses should result in predictable vehicle configurations that support crew and vehicle survival. 

The need for robustness is somewhat bound by the overall goals and mission scenarios that define the system 
and its operation. For a given spacecraft, a set of reference missions and configurations defines cases in which the 
vehicle is expected to either complete or abort the mission. Robustness should be provided to support mission 
execution within the expected bounds (including off-nominal scenarios) and to support mission abort or early 
termination once the defined criteria have been met. Robustness beyond that needed for these cases may not be 
warranted. 

E. Situation Awareness 

Situation Awareness (SA) is the ability to perceive the state of the vehicle and its operational environment, to 
understand that state, and to project the future state based on that understanding. If systems do not inherently 
support SA, additional operator tools and techniques may be required to provide this insight and understanding. 
This may drive additional operations cost and infrastructure such as facility changes, procedures, training, or even 
additional flight control team staffing. The inability to identify specific anomalies in some scenarios may increase 
risks to mission, crew and vehicle. As a result, some activities or objectives may be disallowed when SA cannot be 
maintained. 

Situational awareness should be assured through appropriate telemetry and caution and warning messages which 
allow unambiguous detection and verification of all nominal and off-nominal events. Appropriate sensor locations 
and quantities, as well as telemetry display/downlink capabilities should allow the operator to verify automatically 
generated cues. Simple indications to the operator should be provided for failures with widespread vehicle impacts. 
No false positive or false negative failure indications should be provided to the operator. 

A balanced approach should be taken in assessing situation awareness. Maintaining SA requires the operator to 
have an overall understanding of the system’s state, capabilities and environment. Too much data can make this 
understanding almost as difficult to maintain as can too little data. 

F. Control 

Control measures the degree and difficulty with which the operator can direct the system’s performance during 
operation. This includes not only the availability of all of the control capabilities to appropriately configure the 
system, but also the level of control that the operator must exercise. Use of low level commands - those that control 
individual items at a fine level - may be necessary at times to accomplish specific needs. However, reliance on only 
these low level commands can result in high operator workload because each component must be individually 
configured to accomplish a goal. Higher level commands - those that cause the system to perform multiple steps to 
achieve a predefined configuration - can greatly reduce the level of difficulty in operating the system. Accordingly, 
one effective measure of control is the average count of the number of commands required to implement desired 
courses of action. 
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Ineffective commanding capabilities may require the development of additional ground-based software tools to 
support the configuration management, processing, and issuance of commands in an effective manner. Additional 
procedures may be required to support the configuration and processing of commands. Additional training is 
required to enable operators to use these tools and procedures. All of these add to the infrastructure, cost and time 
associated with controlling the spacecraft. 

Command capabilities should allow the operator to control vehicle functions by setting goals and providing 
decisions when queried. Once these goals and decisions have been provided by the crew, the vehicle implements 
them with little or no additional work required on the part of the crew. Routine functions (those that always involve 
the same steps executed in the same order) should be automated. Where appropriate, low-level commands should 
still be provided to allow for effective operations in off-nominal situations. 

The system should operate and respond in a repeatable, predictable manner to each command. The operator 
should have control over the execution of automated capabilities, allowing him/her to proactively prevent or 
reactively terminate the execution of inappropriate actions. The operator should have the capability to correct the 
vehicle configuration when automation either fails to do so or places the vehicle in an undesirable configuration. 

Automation may be applied to address some control needs, but automation may also create other operability 
challenges. In general, automation of well understood operations is achievable and operationally desirable. 
However, automation of actions or responses to scenarios that are not well understood can make operations more 
difficult. Where automation functions must be monitored by operators, halted as required, and replaced by operator 
actions, the automation function may be operationally undesirable. Even in well understood scenarios, the 
flexibility to modify automation through the use of reconfigurable scripts, settings, and other flexibility measures is 
highly recommended. 


G. Balancing Operability Themes 

There exists a complex association of individual design characteristics with these operability factors, as 
illustrated in Figure 2. The details of these associations require a more thorough discussion than can be provided in 
this paper alone. 
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Figure 2. Vehicle design characteristic influence on flight operability. 


The complex natures of these influences, and the tendency for some of these themes to conflict, make more complex 
the task of establishing formal analytic techniques for operability assessment. Just as is the case with the Cooper- 
Harper scale, then, it is most prudent to rely on flight operations personnel to perform operability assessments and 
assess these complex interrelationships and conflicts. 
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VI. Spacecraft Flight Operability Assessment Scale Content and Structure 

The flight operability assessment scale borrows elements from both the Cooper-Harper Scale and typical 
program risk assessment scales, both of which are illustrated in Figure 3. The overall structure of the scale, 
including its grading range - from one to ten with one being the most desirable score - is reminiscent of the Cooper- 
Harper Scale’s graphical layout. The more detailed textual criteria included in the scale, however, bear closer 
similarity to risk assessment tools. 



Figure 3. Cooper-Harper (left) and risk assessment (right) scales. 


The scale incorporates three basic elements - a set of operability themes to be evaluated, criteria with which to 
evaluate each characteristic, and a grading scale to normalize the results. The operability themes correspond to the 
six operability themes discussed above - simplicity, margin, flexibility, robustness, situation awareness and control. 
Flight operability criteria, as shown in Figure 4, are posed to categorize assessments of each operability theme: “Can 
the mission be accomplished?” ’’Can it be accomplished within tolerable limits (workload, cost, risk)?” “Can it be 
accomplished within normal limits?” and “To what degree?” These four questions guide the assessor in determining 
which color coded range within the possible 10 scores should be assigned for an operability theme. 



Ideal 

0 

Negligible issues 

2 

Nuisance issues 

3 


Some impact 

4 

Moderate impact 

5 

Significant impact 

6 


Some impact 

* 

Moderate impact 


Significant impact 

9 


Improvement 

Mandatory 


Figure 4. Operability assessment criteria. 
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The rating scale provides the remaining guidance - in the form of operational and program impact statements - 
to select the specific rating within a category. Ratings are expressed in terms meaningful to flight operations 
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personnel and to program management. Each number rating has a specific operational impact statement, as shown 
in Figure 5. More generalized program impact statements are mapped to ranges of rating values as well. To provide 
clear guidance regarding application of these ratings to each of the six operability factors, a set of customized 
flowcharts are provided. 
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10 
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Figure 5. Operability grading criteria. 


These elements - the operability themes, criteria and grading scale, are integrated into a single graphical depiction as 
shown in Figure 6. In each case, both a numeric result and a textual description of strengths and deficiencies is 
given. More detailed guidance for each of the six operability themes is provided in a customized version of this 
graphic for the theme of interest. These more detailed graphics are included in the appendix to this paper. 
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Operations are difficult mission objectives may remain at nsk 
even after additional investments (manpower, procedures, 
facilities, etc.) are made 








7 

Operational challenges reduce mission capability and degree 
of mission success by preventing some objectives 







Mission is at risk. 

Operational impacts will exceed the 

8 

Operational challenges put mission success at risk No 

operational techniques are available to mitigate risk 







capabilities of either the operations 
community or the entire program 

9 

Operational challenges increase risk of loss of crew or 
vehicle. No operational techniques are available to mitigate nsk 
while preserving mission content 








10 

Operationally unsafe or unachievable 







Not operable. 






Grading Scale 

Figure 6. Integrated elements of the Spacecraft Flight Operability Scale. 
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The scale is best applied to assessment of specific flight systems or spacecraft functions, such as Guidance, 
Navigation and Control (GNC) or attitude control in specific flight phases, such as ascent or docking. The scale 
allows the reviewer to summarize the operational impacts of one or multiple design features of those systems in 
those operational scenarios. Application of this scale to individual subsystem components such as Line Replaceable 
Units (LRUs) is generally not recommended, as the operability aspects of a system involve more than just the LRU. 
The software supporting that LRU, the user interface displays providing command and control for the subsystem, 
and the interrelationships of that unit with other subsystem components all have a direct impact on the flight 
operability. Capabilities and issues associated with each LRU can, however, be factored into the subsystem-level 
assessment of flight operability. 

It is important to address not only the system design itself, but also the test and verification strategy as part of 
system development and delivery. While an initial design may indicate that margins exist, that a system has 
adequate redundancy, or that the system performs to a given specification, none of these characteristics are truly 
known unless the system is appropriately tested. The test and verification criteria should be inspected as a part of 
operability assessment to ensure that design goals are met. 


VII. Initial Experiences in Applying the Scale 

The Mission Operations Directorate at NASA JSC has begun using this scale as a tool for both operational 
vehicles such as the Space Shuttle and for new vehicle designs such as those developed under the Constellation 
Program. To date, assessments of 46 Space Shuttle Orbiter subsystems have been completed for ascent, orbit and 
entry scenarios. An integrated review of these results is underway to ensure consistency. Assessment of currently 
operational vehicles serves to calibrate the assessment scale by identifying areas in which clearer guidance must be 
given to ensure consistent evaluation results regardless of the person performing the evaluation or the system under 
evaluation. 


Operability Theme 

Score 

Description 

Operational Impact 

Program Impact 

Simplicity 

5 

Multiple nominal and off nominal 
procedures as well as operational 
workarounds to disable and release 
dampers indicate inherent undesirable 
complexity in the system. 

Complexity increases operator w orkload, requiring 
additional tools and techniques (procedures, 
constraints, etc ). 

Some mission objectives 
may be at risk. 

Margin 

8 

Little margin is available in the APDS 
hooks. Single hook out cases drive the 
need for significant system workarounds 
(PMA hooks, FR constraints etc). 

Single point jam on the ball screw 
mechanism could lead to loss of 
mission. 

^adequate margin induces risk of toss of mission. 

Mission is at risk. 

Flexibility 

3 

The semi-automatic docking sequence 
allows for much greater system 
flexibility but also poses issues with 
added training due to it's complexity. 

Functions enabling flexibility induce additional operator 
w orkload w ithin reasonable limits. 

Mission can be 
accomplished. 

Robustness 

8 

Capture latches require manual 
reconfiguring after the first failure to 
return to a nominal configuration. A 
single point jam on the ball screw 
mechanism can cause loss of mission. 

friability to recover sufficient functionality increases 
risk of toss of mission. 

Mission is at risk. 

Situation 

Awareness 

3 

In general, enough insight into the health 
and operation of the docking system is 
available to MCC. Some coordination 
with crew to attain crew only insight (A7 
panel lights add to MCC workload. 

Required effort to maintain Situational Aw areness 
results in minor w orkload impacts. 

Mission can be 
accomplished. 

Control 

3 

Lack of ground control capability limits 
MCC ability to operate the docking 
system in off nominal situations. 

Command & control interfaces and tasks impact 
workload but remain in reasonable limits 

Mssion can be 
accomplished. 


Figure 7. Sample of a completed flight operability assessment for Space Shuttle docking system during orbit operations. 
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An example of an assessment of a Space Shuttle subsystem in this format is shown in Figure 7. The comments 
shown m this example illustrate many of the typical operational impacts of spacecraft design. The subsystem scores 
relatively well in the categories of flexibility, situation awareness and control, though some limitations capabilities 
are noted. However, the system’s scores in simplicity, margin and robustness scores reflect the significant 
operational impacts of even a single failure in the subsystem. 

Initial use of this scale within the operations community has yielded encouraging results. Evaluators find the scale 
easy to use, and the resultmg evaluations quickly identify and isolate operability issues within specific subsystems 
and scenarios. For user convenience, a pre-formatted spreadsheet form is used to assist the evaluator. 


VIII. Conclusion 

There is more work to be done in the development and industry-wide adoption of formal flight operability 
expectations. There has been good success in initial steps to isolating the major criteria that define flight operability. 
Early efforts to apply the scale to operational programs demonstrate that the scale can be applied across many 
different spacecraft systems, and that the evaluation process extracts useful feedback regarding design 
characteristics and operability impacts. Through the continued application of this process to existing and future 
programs, it is hoped that the scale and supporting material can be both matured and disseminated to a wider 
audience within the aerospace community. 


Appendix 

Graphical desictions of the Spaceflight Operability Assessment Scale and related detailed rating guidance for 
mdivdual operability themes are provided velow 



Operational 

Impact 


CO 

1 1 


id?; 

I = = ? 




i I 


Program 

Impact 


Excellent operations capabilities 







ODerationallv 

desirable. 

2 Negligible operational challenges that can be handled with 
no noticeable impact to operations feasibility or cost 







Mission can be j 

accomplished 

^ Operational challenges cause noticeable nuisances to 

the operator, but can be handled with little impact to 
operations feasibility or cost. 







Minimal operational impacts 
can be handled within 
existing infrastructure and 
budget with negligible 
workload impacts. 

4 Operations are difficult and incur sianif icant one time costs 
(manpower, facilities, products, etc.) to ensure mission 
success. Some mission objectives may not be achieved 







Some mission 
objectives may be at 
risk. 

Operational impacts will 
change infrastructure 
requirements, cost 
allocations, work 
prioritization, etc. from the 
baseline operations plan. 

0 Operations are difficult and incur sianif icant recurrina costs 
(manpower, facilities, products, etc ) to ensure mission 
success Some mission objectives may not be achieved 







0 Operations are difficult, mission objectives may remain at 
risk even after additional investments (manpower, 
procedures, facilities, etc.) are made 







■y Operational challenges reduce mission capability and 

degree of mission success by preventing some objectives 







Mission is at risk. 

Operational impacts will 
exceed the capabilities of 
either the operations 
community or the entire 
program. 

g Operational challenges put mission success at risk No 

operational techniques are available to mitigate risk. 







0 Operational challenges increase risk of loss of crew or 
vehicle. No operational techniques are available to mitigate 
risk while preserving mission content. 







•j q Operationally unsafe or unachievable 







Not werable. l 


System & mission design 


*Operability assessment is performed for a specific reference mission or scenario 

Figure 8. Spacecraft Flight Operability Assessment Scale. 
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System and mission design 


Operationally acceptable 

Required operational techniques are 
as simple as practical. 

Functions, interfaces and tasks require lowest 
practical operator workload and infrastructure. 

1 

Minor complexity may cause nuisances but 
does not impact operator workload. 

2 

Minor complexity increases operator workload, 
but workload remains in reasonable limits. 

3 

Deficiencies warrant 
improvement 

Complexity drives additional cost in 
developing operations infrastructure 
and may risk los of some mission 
objectives. 

Complexity increases operator workload and 
requires additional tools to support the 
operator. 

4 

Complexity increases operator workload, 
requiring additional tools and techniques 
(procedures, constraints, etc.). 

5 

Complexity drives infrastructure costs, but risk 
to some mission objectives remains. 

6 

Deficiencies require 
improvement 

Complexity in system functions, 
interfaces, or interdependencies 
induce significant operational cost 
(excessive procedures, constraints, 
training, etc) and may reduce mission 
success. 

Complexity prevents accomplishment of some 
mission objectives. 

7 

Complexity drives operational constraints that 
threaten mission success. 

8 

Complexity drives operational constraints that 
increase risk of loss of crew or vehicle. 

9 


Improvement Mandatory 

Complexity severely impacts 
operations 


Complexity results in unacceptable risk to the 
mission vehicle and crew 


Figure 9. Detailed guidance for assessing simplicity. 


Operationally acceptable 

Useful positive margin is available, 
enabling simple planning and 
providing advantage in off-nominal 
cases. 

Significant useful margin is available in most or 
all cases. 

1 

Some useful margin is available in most cases. 

2 

Slight useful margin is available in most cases. 

3 

Deficiencies warrant 
improvement 

Zero or negative margin in one or 
more categories may cause non- 
critical impacts (including loss of 
some mission objectives) either 
during nominal operations or after 1 st 
failure 

Lack of margin drives additional operations 
infrastructure (facility capabilities). 

4 

Lack of margin drives additional infrastructure 
and processes (facility capabilities, analysis 
and procedures). 

5 

Additional infrastructure and processes cannot 
fully mitigate risk to mission objectives. 

6 

Deficiencies require 
improvement 

Lack of margin in lor more categories 
may cause critical impacts (potential 
loss of mission, crew or vehicle) 
either during nominal operations or 
after 1 st failure. 

Inadequate margin prevents accomplishment 
of some mission objectives. 

7 

Inadequate margin induces risk of loss of 
mission. 

8 

Inadequate margin induces risk of loss of 
crew/vehicle. 

9 

Improvement Mandatory 

Lack of margin will cause critical 
impacts (potential loss of crew or 
vehicle) 

Inadequate margin is available to execute 
mission 

10 



Figure 10. Detailed guidance for assessing margin. 
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Operationally acceptable 

Flexibility is inherently available in the 
system design and interfaces 
Flight systems may be easily 
reconfigured to account for changed 
conditions or to incorporate new 
operational techniques. 

Flexibility is seamlessly provided without 
requiring additional operator action. 

T 

Functions enabling flexibility induce nuisances 
but do not impact operator workload. 

2 

Functions enabling flexibility induce additional 
operator workload within reasonable limits. 

3 

Deficiencies warrant 
improvement 

Necessary flexibility may be 
achieved, but only with significant 
additional investments. 

Additional tools and infrastructure must be 
developed to support flexibility (data and 
software reconfiguration, etc.) 

4 

Excessive procedural workarounds and 
processes are required to accommodate the 
lack of inherent system flexibility. 

5 

Infrastructure and procedural workarounds are 
required, but even these do not mitigate all risk 
to some mission objectives. 

6 

Deficiencies require 
improvement 

Required flexibility is not provided and 
will impact mission capabilities. 

Lack of necessary flexibility will result in loss of 
some mission objectives. 

7 

Lack of necessary flexibility induces additional 
risk of loss of mission. 

8 

Lack of necessary flexibility induces additional 
risk of loss of crew or vehicle. 

9 

Improvement Mandatory 

Inflexibility prevents reconfiguration required to 
safely execute missions 

10 


Figure 11. Detailed guidance for assessing flexibility. 


Can system 
handle non-critical 
anomalies without 



Operationally acceptable 

The system makes best possible use 
of remaining functionality after an 
anomaly, remaining capable of 
completing the mission with little or no 
change to the mission plan. 

No further action is required of the operator 
after this reconfiguration 

i 

System functionality is preserved, but non- 
critical activities may be temporarily impacted 
by the recovery process. 

2 

System functionality is preserved, but some 
activities may be interrupted until additional 
manual steps are taken. 

3 

Deficiencies warrant 
improvement 

Manual proactive or reactive 
measures are required to ensure 
adequate system operation after a 
failure. 

Additional operator action is required to 
establish normal function after a failure. 

4 

Operator must manually pre-configure systems 
to ensure proper response to possible failures. 

5 

Manual pre-configuration alone cannot 
completely mitigate risks, some mission 
objectives remain at risk. 

6 

Deficiencies require 
improvement 

Inability to recover necessary 
functionality or inappropriate 
automated recovery functions 
significantly impact mission success 
or crew safety. 

Inability to recover sufficientfunctionality 
prevent completion of some mission 
objectives. 

7 

Inability to recover sufficientfunctionality 
increases risk of loss of mission. 

8 

Inability to recover sufficientfunctionality 
increases risk of loss of crew or vehicle. 

9 


Improvement mandatory 


Inability to properly reconfigure after a single 
failure causes loss of crew or vehicle 


10 


System and mission design 


Figure 12. Detailed guidance for assessing robustness. 
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Operationally acceptable 

Telemetry and caution & warning 
messages allow unambiguous 
detection and verification of all 
nominal and off-nominal events. 

SAis properly maintained in all scenarios with 
no additional operator action required. 

i 

Minor nuisances in SA tools are noticeable but 
do not add to operator workload. 

2 

Required effort to maintain Situational 
Awareness results in minor workload impacts. 

3 

Deficiencies warrant 
improvement 

High operator workload is required to 
maintain situational awareness. Asa 
result, some mission objectives may 
be at risk. 

Additional tools must be developed to achieve 
the necessary level of Situational Awareness 

4 

Additional techniques (procedures, training, 
etc.) must be developed to achieve the 
necessary level of Situational Awareness. 

5 

Even with additional tools and techniques, 
some non-critical conditions cannot be 
effectively identified. 

6 

Deficiencies require 
improvement 

Data or cues required to recognize 
non-critical events (nominal or off- 
nominal) are incorrect or not 
available. 1 ncorrect indications may 
cause the operator to take actions 
that threaten mission success. 

Lack of suitable SA imposes constraints on 
activities and operations, placing mission 
objectives at risk. 

7 

Lack of suitable SA increases risk of loss of 
mission due to potential operator error. 

8 

Lack of suitable SA increases risk of loss of 
crew or vehicle due to potential operator error. 

9 


Improvement Mandatory 

Insufficient insight to execute a 
nominal mission 


Data or cues required to recognize critical 
events (nominal or off-nominal) are incorrect or 
not available Incorrect indications will cause 
the operator to take inappropriate critical 
actions that impact crew/vehicle survival Will 
cause critical impacts to nominal operations 


Figure 13. Detailed guidance for assessing situation awareness. 


Operationally acceptable 

All necessary control and control 
functions a re provided. Required 
operator control tasks are 
appropriate. 

Command interfaces are efficient and do not 
contribute significantly to operator workload. 

i 

Command & control interfaces include some 
nuisances that do not impact workload. 

2 

Command & control interfaces and tasks 
impactworkload but remain in reasonable 
limits. 

3 

Deficiencies warrant 
improvement 

Commands and controls are difficult 
to operate, impacting operator 
workload and inducing additional 
costs. 

Additional infrastructure must be developed to 
support command and control capabilities. 

T 

Extra tools and procedures are required to 
achieve necessary control. 

5 

Extra tools and techniques are required to 
achieve necessary control, but workload 
impacts may impede completion of some 
mission objectives. 

6 

Deficiencies require 
improvement 

Inadequate control interfaces 
and methodologies impact the 
ability to safely and successfully 
complete the mission. 

Insufficient control capability is provided to 
support execution of some mission objectives. 

7 

Insufficient control capability is provided to 
respond to anomalies that risk loss of mission. 

8 

Insufficient control capability is provided to 
respond to anomalies that risk loss of 
crew/vehicle. 

9 

Improvement Mandatory 

Insufficient control capabilities to 
executes nominal mission 

Control interfaces or methodology will cause 
critical impacts to nominal operations (loss of 
crew/vehicle) 

10 



Figure 14. Detailed guidance for assessing control. 
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A Technique for the Assessment of 
Flight Operability Characteristics of 
Human Rated Spacecraft 


Alan Crocker 


NASA Johnson Space Center 


Overview 


The Challenge - defining operability 

Role of Operability Assessment in a Spaceflight 
Program 

Spacecraft Flight Operability Assessment Scale 
Structure 


Initial Application & Lessons Learned 



What is "Flight Operability?" 


(Exactly. That's the problem. There is not a 
formal definition.) 


3 


We face challenges similar to those faced by test 
pilots and aircraft designers 40+ years ago. 


To improve aircraft designs, they needed a way to quantify the pilot's 
needs and criteria for aircraft handling qualities ("Stick and rudder" 
feel) 

George Cooper and Robert Harper - a test pilot and a test engineer - 
devised a scale to meet this need. 

- The scale evolved over a period of 12 years to become the modern version (1957- 
1969) 


Today, the Cooper-Harper Scale is the standard accepted means for 
specification of aircraft handling 
— Even Constellation has a Cooper-Harper rating requirement. 


The human spaceflight community needs a similar method to clearly 
characterize flight operability - and communicate operability issues - as 
we execute the design process. 




Goal: Establish a framework for assessing operability 
concerns. 

• Define general operational expectations and criteria. 

- Describe the key operations concerns. 

- Establish criteria for evaluation of those concerns. 

- Map evaluation results back to impacts on program. 

• Incorporate this framework into program systems 
engineering process and schedules. 

- Use this framework throughout the design and review process to 
organize and justify our ops inputs. 


Operability assessment fit into a larger set of 
processes. 


• Operability issues are linked to 
safety, reliability, performance, 
etc. 

— There are other tools available 
to assess these topics 

• An operability assessment tool 
should not replace other 
assessment tools, but rather 
add to the toolset. 


Affordability 

Maintainability 

Sustainability 

Operability 

Safety Reliabilitv 


Method of Program 

Ops Influence Phase 


Formal definitions and criteria for operability can 
benefit the Program throughout its life cycle. 



Operations concept 
definition includes 
factors that impact 
flight operations 


System 

requirements 

include 

operability 

criteria 


Operability criteria 
provide clear 
guidance for 
design 

implementation 


Formal 

operability 

assessments 

ensure 

requirements 

compliance 


Operability 

assessments 

applied to 

proposed 

design 

changes and 

upgrades/ 










There have been several attempts to define 

operability... 



j 

JSC 

JSC 

Flight 

Operations 

Human Space 

Improvement 

Systems 

Team (FOIT) 

Operational 

Report 

Design Criteria 


Manual 


(John 

Commonsense) 


JPL 

Design 

Principles 

Document 


ESA 

ECSS 
Standard 
on Space 
Segment 
Operability 


...but it remains difficult to establish formal 
requirements that completely reflect these needs. 
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Why not use an existing scale like... 


.Cooper Harper? 

- Assessment requires availability of a simulator to 
perform evaluation. 

- Only directly address the real-time aspect of 
mission operations ("Fly" vs. "Plan-Train-Fly"). 



.or a risk matrix? 

- Too generalized to completely reflect operability 
drivers. 



But we can use ideas from both in building an 
operability scale. 


There are three key elements to this framework: 


Operability Factors 

• Capture the general factors that drive ops complexity 

• Include description of desired characteristics in each theme 


Criteria 

• General questions that characterize operations impact 

• Can be customized for each theme 

Grading scale 

• Define the range of possible scores and their implications to ops and the 
program 

• The resulting grades must have meaning for both the operations 
community and the program management community. 


These elements are applied to a system design for a specific design 
reference mission or task 
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Six operability factors capture the range of operational 
concerns. 


* Simplicity 

* Margin — 

* Flexibility 

* Robustness 


Commonality and consistency 
Simple functions and interfaces 
Simple tasks 

Performance margin 
Resource margin 

Environmental tolerance (temperature, radiation, etc.) 
Easy reconfiguration 

Ability to make minor updates (limits, control gains, etc.) 
Ability to upgrade through life cycle 

Fail operational 
Graceful degradation 

Appropriate automation time-critical reconfiguration 


* Situational 
Awareness 


telemetry and caution & warning 
Sensor locations and quantities 
Simple indications for the operator 




Controllability 


Command capabilities 

Control of automated capabilities 

Systems operate in a repeatable, predictable manner. 
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There are many complex relationships between design 
characteristics, operability factors, and resulting program impact. 


Vehicle Design 
Characteristics 


Environment 
Tolerance 

Consumable 
resource storage 
capacity 

Renewable 
resource storage 
capacity 

Performance 
Analysis Data 
Availability 


System Architecture 
& Connectivity 

Redundancy 
& Reliability 

Hazards 

Automation 
Command 
Interfaces 
Instrumentation 

Communication 
Bandwidth 

Caution & Warning 




Operability 

Impacts 



Margin 



Flexibility 


Simplicity 


Robustness 

Control 


Situation 

Awareness 




Operational 

Response 


4 


Operational 

Impact 




Program 

Impact 



Operational 

Constraints 


Reactive 

Reconfiguration 


Proactive 

Reconfiguration 



Life Cycle 
Cost 

& Mission 
Success 

Fit Procedure 
Quantity & 
Complexity 

(risk, time, cost) 

Analysis Task 
Quantity & 
Complexity 

(risk, time, cost) 

Analysis Tool 
Needs (cost) 

Preflight 
Planning 
complexity 

(risk, time, cost) 

Training Needs 

(time, cost) 
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General criteria apply to all themes 



Ideal 

1 

Negligible issues 

2 

Nuisance issues 

3 


Some impact 

4 

Moderate impact 

5 

Significant impact 

6 


Some impact 

7 

Moderate impact 

8 

Significant impact 

9 


Unsafe I 10 


System and mission design 


Ideal 


Below this point, 
anticipated capabilities or 
budget levels are not 
supportable 
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The grading scale translates results into ops and 
program impacts. 


Operational Impact Programmatic impact 


1 

Excellent operations capabilities 

Operationallv desirable. 

2 

Negligible operational challenges that can be handled with no 
noticeable impact to operations feasibility or cost 

Mission can be accomplished 

Minimal operational impacts can be 
handled within existing infrastructure 
and budget with negligible workload 
impacts. 

3 

Operational challenges cause noticeable nuisances to the 
operator, but can be handled with little impact to operations 
feasibility or cost. 

4 

Operations are difficult and incur sianificant one time costs 

(manpower, facilities, products, etc.) to ensure mission success. 
Some mission objectives may not be achieved. 

Some mission objectives mav be at 
risk. Operational impacts will change 
infrastructure requirements, cost 
allocations, work prioritization, etc. 
from the baseline operations plan. 

5 

Operations are difficult and incur sianificant recurrinq costs 

(manpower, facilities, products, etc.) to ensure mission success. 
Some mission objectives may not be achieved. 

6 

Operations are difficult, mission objectives may remain at risk 

even after additional investments (manpower, procedures, facilities, 
etc.) are made. 

7 

Operational challenges reduce mission capability and degree of 
mission success by preventing some objectives 

Mission is at risk. i 

Operational impacts will exceed the 
capabilities of either the operations 
community or the entire program. 

8 

Operational challenges put mission success at risk. No 

operational techniques are available to mitigate risk. 

9 

Operational challenges increase risk of loss of crew or vehicle. 

No operational techniques are available to mitigate risk while 
preserving mission content. 

10 

Operationally unsafe or unachievable 

Not operable. 



Themes, criteria and grading scale are integrated 
into an evaluation table. 


Criteria 
\ 



Operational 

Impact 


(/> 
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Themes 


Program Impact 


1 

Excellent operations capabilities 







Operationally desirable. 

2 

Negligible operational challenges that can be handled with no 
noticeable impact to operations feasibility or cost 







Mission can be accomplished j 

Minimal operational impacts can be 

3 

Operational challenges cause noticeable nuisances to the 
operator, but can be handled with little impact to operations 
feasibility or cost. 







handled within existing infrastructure and 
budget with negligible workload impacts. 

4 

Operations are difficult and incur significant one time costs 
(manpower, facilities, products, etc.) to ensure mission success. 
Some mission objectives may not be achieved. 







Some mission objectives may be at 
risk. 

Operational impacts will change 

5 

Operations are difficult and incur significant recurring costs 
(manpower, facilities, products, etc.) to ensure mission success 
Some mission objectives may not be achieved. 







infrastructure requirements, cost 
allocations, work prioritization, etc. from the 
baseline operations plan. 

6 

Operations are difficult, mission objectives may remain at risk 
even after additional investments (manpower, procedures, 
facilities, etc.) are made. 








7 

Operational challenges reduce mission capability and degree 
of mission success by preventing some objectives 







Mission is at risk. 

Operational impacts will exceed the 

8 

Operational challenges put mission success at risk. No 
operational techniques are available to mitigate risk. 







capabilities of either the operations 
community or the entire program. 

9 

Operational challenges increase risk of loss of crew or 
vehicle. No operational techniques are available to mitigate risk 
while preserving mission content. 








10 

Operationally unsafe or unachievable 







Not operable. 


Grading Scale 
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Operational 

Impact 
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Program 

Impact 


-j Excellent operations capabilities 







Operationally 

desirable. 

2 Negligible operational challenges that can be handled with 
no noticeable impact to operations feasibility or cost 







Mission can be 
accomplished 

3 Operational challenges cause noticeable nuisances to 

the operator, but can be handled with little impact to 
operations feasibility or cost. 







Minimal operational impacts 
can be handled within 
existing infrastructure and 
budget with negligible 
workload impacts. 

^ Operations are difficult and incur sianificant one time costs 
(manpower, facilities, products, etc.) to ensure mission 
success. Some mission objectives may not be achieved. 







Some mission 
objectives may be at 
risk. 

Operational impacts will 
change infrastructure 
requirements, cost 
allocations, work 
prioritization, etc. from the 
baseline operations plan. 

g Operations are difficult and incur sianificant recurring costs 
(manpower, facilities, products, etc.) to ensure mission 
success Some mission objectives may not be achieved. 







g Operations are difficult, mission objectives may remain at 
risk even after additional investments (manpower, 
procedures, facilities, etc.) are made. 







y Operational challenges reduce mission capability and 

degree of mission success by preventing some objectives 







Mission is at risk. 

Operational impacts will 
exceed the capabilities of 
either the operations 
community or the entire 
program. 

g Operational challenges put mission success at risk. No 

operational techniques are available to mitigate risk. 







0 Operational challenges increase risk of loss of crew or 
vehicle. No operational techniques are available to mitigate 
risk while preserving mission content. 







<| q Operationally unsafe or unachievable 







Not operable. 


l^^ystem^Miiission^^des^ 


*Operability assessment is performed for a specific reference mission or scenario 
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For each operability theme, more detailed guidance is given 

(Example - margin) 



Operationally acceptable 

Useful positive margin is available, 
enabling simple planning and 
providing advantage in off-nominal 
cases. 

Significant useful margin is available in most or 
all cases. 

1 

Some useful margin is available in most cases. 

2 

Slight useful margin is available in most cases. 

3 

Deficiencies warrant 
improvement 

Zero or negative margin in one or 
more categories may cause non- 
critical impacts (including loss of 
some mission objectives) either 
during nominal operations or after 1 st 
failure. 

Lack of margin drives additional operations 
infrastructure (facility capabilities). 

4 

Lack of margin drives additional infrastructure 
and processes (facility capabilities, analysis 
and procedures). 

5 

Additional infrastructure and processes cannot 
fully mitigate risk to mission objectives. 

6 

Deficiencies require 
improvement 

Lack of margin in lor more categories 
may cause critical impacts (potential 
loss of mission, crew or vehicle) 
either during nominal operations or 
after 1 st failure. 

Inadequate margin prevents accomplishment 
of some mission objectives. 

7 

Inadequate margin induces risk of loss of 
mission. 

8 

Inadequate margin induces risk of loss of 
crew/vehicle. 

9 

Improvement Mandatory 

Lack of margin will cause critical 
impacts (potential loss of crew or 
vehicle) 

Inadequate margin is available to execute 
mission. 

10 
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Initial Application & Lessons Learned 

Establishing a baseline for future assessments 


Executed an initial operability assessment 
of Space Shuttle flight systems 

- This was the first real exercise, using well 
understood design and a wealth of 
operational experience. 

- Provided guidance and criteria for 
assigning operability scores, but recognized 
that this first attempt would show 
variations form reviewer to reviewer 

Goals 

- Identify the major gaps in defined criteria 

- Explore possible interpretations of the 
criteria as written 

- Begin working towards a consistent 
approach for all technical disciplines. 



Scope of Space Shuttle Assessment 

• Six major flight systems (with a total of 
46 subsystems contained therein) 
-Communications 
- ECLSS 
-EPS 
-GNC 

— Mechanical 
— Propulsion 


• Three major mission scenarios 


— Ascent 
-Orbit 

— Entry 



High degree of 
similarity 


Initial Application 

Example - Evaluation of Shuttle Docking system for on-orbit operations 


Operability Theme 

Score 

Description 

Operational Impact 

Program Impact 

Simplicity 

5 

Multiple nominal and off nominal 
procedures as well as operational 
workarounds to disable and release 
dampers indicate inherent undesirable 
complexity in the system. 

Complexity increases operator workload, requiring 
additional tools and techniques (procedures, 
constraints, etc.). 

Some mission objectives 
may be at risk. 

Margin 

8 

Little margin is available in the APDS 
hooks. Single hook out cases drive the 
need for significant system workarounds 
(PMA hooks, FR constraints etc). 

Single point jam on the ball screw 
mechanism could lead to loss of 
mission. 

Inadequate margin induces risk of loss of mission. 

Mission is at risk. 

Flexibility 

3 

The semi-automatic docking sequence 
allows for much greater system 
flexibility but also poses issues with 
added training due to it's complexity. 

Functions enabling flexibility induce additional operator 
workload within reasonable limits. 

Mission can be 
accomplished. 

Robustness 

8 

Capture latches require manual 
reconfiguring after the first failure to 
return to a nominal configuration. A 
single point jam on the ball screw 
mechanism can cause loss of mission. 

Inability to recover sufficient functionality increases 
risk of loss of mission. 

Mission is at risk. 

Situation 

Awareness 

3 

In general, enough insight into the health 
and operation of the docking system is 
available to MCC. Some coordination 
with crew to attain crew only insight (A7 
panel lights add to MCC workload. 

Required effort to maintain Situational Aw areness 
results in minor w orkload impacts. 

Mission can be 
accomplished. 

Control 

3 

Lack of ground control capability limits 
MCC ability to operate the docking 
system in off nominal situations. 

Command & control interfaces and tasks impact 
workload but remain in reasonable limits. 

Mission can be 
accomplished. 
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Lessons Learned 


Defining flight operability is non-trivial. But Important. 

Development - and adoption - of a formal technique 
will take time. 

So far... 

- General acceptance of operability theme definitions in 
flight operations community 

- Evaluation process generates findings that can benefit 
future programs 

- Well received by program and development communities 



